Navigating the Complexities of Artificial Intelligence-Enabled Real-World Data Collection for Oncology Pharmacovigilance.

This new editorial discusses the promise and challenges of successful integration of natural language processing methods into electronic health records for timely, robust, and fair oncology pharmacovigilance.

Best Practices and Recommendations for Research Using Virtual Real-Time Data Collection: Protocol for Virtual Data Collection Studies.

BACKGROUND: The COVID-19 pandemic and the subsequent need for social distancing required the immediate pivoting of research modalities. Research that had previously been conducted in person had to pivot to remote data collection. Researchers had to develop data collection protocols that could be conducted remotely with limited or no evidence to guide the process. Therefore, the use of web-based platforms to conduct real-time research visits surged despite the lack of evidence backing these novel approaches. OBJECTIVE: This paper aims to review the remote or virtual research protocols that have been used in the past 10 years, gather existing best practices, and propose recommendations for continuing to use virtual real-time methods when appropriate. METHODS: Articles (n=22) published from 2013 to June 2023 were reviewed and analyzed to understand how researchers conducted virtual research that implemented real-time protocols. "Real-time" was defined as data collection with a participant through a live medium where a participant and research staff could talk to each other back and forth in the moment. We excluded studies for the following reasons: (1) studies that collected participant or patient measures for the sole purpose of engaging in a clinical encounter; (2) studies that solely conducted qualitative interview data collection; (3) studies that conducted virtual data collection such as surveys or self-report measures that had no interaction with research staff; (4) studies that described research interventions but did not involve the collection of data through a web-based platform; (5) studies that were reviews or not original research; (6) studies that described research protocols and did not include actual data collection; and (7) studies that did not collect data in real time, focused on telehealth or telemedicine, and were exclusively intended for medical and not research purposes. RESULTS: Findings from studies conducted both before and during the COVID-19 pandemic suggest that many types of data can be collected virtually in real time. Results and best practice recommendations from the current protocol review will be used in the design and implementation of a substudy to provide more evidence for virtual real-time data collection over the next year. CONCLUSIONS: Our findings suggest that virtual real-time visits are doable across a range of participant populations and can answer a range of research questions. Recommended best practices for virtual real-time data collection include (1) providing adequate equipment for real-time data collection, (2) creating protocols and materials for research staff to facilitate or guide participants through data collection, (3) piloting data collection, (4) iteratively accepting feedback, and (5) providing instructions in multiple forms. The implementation of these best practices and recommendations for future research are further discussed in the paper. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): DERR1-10.2196/53790.

Describing diversity of real world data sources in pharmacoepidemiologic studies: The DIVERSE scoping review.

PURPOSE: Real-world evidence (RWE) is increasingly used for medical regulatory decisions, yet concerns persist regarding its reproducibility and hence validity. This study addresses reproducibility challenges associated with diversity across real-world data sources (RWDS) repurposed for secondary use in pharmacoepidemiologic studies. Our aims were to identify, describe and characterize practices, recommendations and tools for collecting and reporting diversity across RWDSs, and explore how leveraging diversity could improve the quality of evidence. METHODS: In a preliminary phase, keywords for a literature search and selection tool were designed using a set of documents considered to be key by the coauthors. Next, a systematic search was conducted up to December 2021. The resulting documents were screened based on titles and abstracts, then based on full texts using the selection tool. Selected documents were reviewed to extract information on topics related to collecting and reporting RWDS diversity. A content analysis of the topics identified explicit and latent themes. RESULTS: Across the 91 selected documents, 12 topics were identified: 9 dimensions used to describe RWDS (organization accessing the data source, data originator, prompt, inclusion of population, content, data dictionary, time span, healthcare system and culture, and data quality), tools to summarize such dimensions, challenges, and opportunities arising from diversity. Thirty-six themes were identified within the dimensions. Opportunities arising from data diversity included multiple imputation and standardization. CONCLUSIONS: The dimensions identified across a large number of publications lay the foundation for formal guidance on reporting diversity of data sources to facilitate interpretation and enhance replicability and validity of RWE.

A practical guide to cross-cultural and multi-sited data collection in the biological and behavioural sciences.

Researchers in the biological and behavioural sciences are increasingly conducting collaborative, multi-sited projects to address how phenomena vary across ecologies. These types of projects, however, pose additional workflow challenges beyond those typically encountered in single-sited projects. Through specific attention to cross-cultural research projects, we highlight four key aspects of multi-sited projects that must be considered during the design phase to ensure success: (1) project and team management; (2) protocol and instrument development; (3) data management and documentation; and (4) equitable and collaborative practices. Our recommendations are supported by examples from our experiences collaborating on the Evolutionary Demography of Religion project, a mixed-methods project collecting data across five countries in collaboration with research partners in each host country. To existing discourse, we contribute new recommendations around team and project management, introduce practical recommendations for exploring the validity of instruments through qualitative techniques during piloting, highlight the importance of good documentation at all steps of the project, and demonstrate how data management workflows can be strengthened through open science practices. While this project was rooted in cross-cultural human behavioural ecology and evolutionary anthropology, lessons learned from this project are applicable to multi-sited research across the biological and behavioural sciences.

Deriving life-course residential histories in brain bank cohorts: A feasibility study.

INTRODUCTION: The exposome is theorized to interact with biological mechanisms to influence risk for Alzheimer's disease but is not well-integrated into existing Alzheimer's Disease Research Center (ADRC) brain bank data collection. METHODS: We apply public data tracing, an iterative, dual abstraction and validation process rooted in rigorous historic archival methods, to develop life-course residential histories for 1254 ADRC decedents. RESULTS: The median percentage of the life course with an address is 78.1% (IQR 24.9); 56.5% of the sample has an address for at least 75% of their life course. Archivists had 89.7% agreement at the address level. This method matched current residential survey methodology 97.4% on average. DISCUSSION: This novel method demonstrates feasibility, reproducibility, and rigor for historic data collection. To our knowledge, this is the first study to show that public data tracing methods for brain bank decedent residential history development can be used to better integrate the social exposome with biobank specimens. HIGHLIGHTS: Public data tracing compares favorably to survey-based residential history. Public data tracing is feasible and reproducible between archivists. Archivists achieved 89.7% agreement at the address level. This method identifies residences for nearly 80% of life-years, on average. This novel method enables brain banks to add social characterizations.

Capture, Coding, and Reporting of Health Care and Medicine Information in Australian Workers' Compensation Systems.

OBJECTIVE: This study aims to characterize the approaches to collecting, coding, and reporting health care and medicines data within Australian workers' compensation schemes. METHODS: We conducted a cross-sectional survey of data and information professionals in major Australian workers' compensation jurisdictions. Questionnaires were developed with input from key informants and a review of existing documentation. RESULTS: Twenty-five participants representing regulators (40%) and insurers (60%) with representation from all Australian jurisdictions were included. Health care and medicines data sources, depth, coding standards, and reporting practices exhibited significant variability across the Australian workers' compensation schemes. CONCLUSIONS: Substantial variability exists in the capture, coding, and reporting of health care and medicine data in Australian workers' compensation jurisdictions. There are opportunities to advance understanding of medicines and health service delivery in these schemes through greater harmonization of data collection, data coding, and reporting.

Lessons Learned From Telephone-Based Data Collection for Health and Demographic Surveillance Systems During the COVID-19 Pandemic in Indonesia.

The Sleman Health and Demographic Surveillance System (HDSS) is a longitudinal survey held routinely since 2014 to collect demographic, social, and health changes in Sleman Regency, Special Region of Yogyakarta, Indonesia. During the COVID-19 pandemic in Indonesia, we needed to adjust our method of conducting data collection from in-person to telephone interviews. We describe the Sleman HDSS data collection strategy used and the opportunities it presented. First, the Sleman HDSS team completed a feasibility study and adjusted the standard operational procedures to conduct telephone interviews. Then, the Sleman HDSS team collected data via a telephone interview in September-October 2020. Ten interviewers were equipped with an e-HDSS data collection application installed on an Android-based tablet to collect data. The sample targeted was 5,064 households. The telephone-based data collection successfully interviewed 1,674 households (33% response rate) in 17 subdistricts. We changed the data collection strategy so that the Sleman HDSS could still be conducted and we could get the latest data from the population. Compared to in-person interviewing, data collection via telephone was sufficiently practical. The telephone interview was a safe and viable data collection method. To increase the response rate, telephone number activation could be checked, ways of building rapport could be improved, and engagement could be improved by using social capital.

Leveraging Ethnic Backgrounds to Improve Collection of Race, Ethnicity, and Language Data.

INTRODUCTION: Healthcare disparities may be exacerbated by upstream incapacity to collect high-quality and accurate race, ethnicity, and language (REaL) data. There are opportunities to remedy these data barriers. We present the Denver Health (DH) REaL initiative, which was implemented in 2021. METHODS: Denver Health is a large safety net health system. After assessing the state of REaL data at DH, we developed a standard script, implemented training, and adapted our electronic health record to collect this information starting with an individual's ethnic background followed by questions on race, ethnicity, and preferred language. We analyzed the data for completeness after REaL implementation. RESULTS: A total of 207,490 patients who had at least one in-person registration encounter before and after the DH REaL implementation were included in our analysis. There was a significant decline in missing values for race (7.9%-0.5%, p < .001) and for ethnicity (7.6%-0.3%, p < .001) after implementation. Completely of language data also improved (3%-1.6%, p < .001). A year after our implementation, we knew over 99% of our cohort's self-identified race and ethnicity. CONCLUSIONS: Our initiative significantly reduced missing data by successfully leveraging ethnic background as the starting point of our REaL data collection.

A comparative study of clinical trial and real-world data in patients with diabetic kidney disease.

A growing body of research is focusing on real-world data (RWD) to supplement or replace randomized controlled trials (RCTs). However, due to the disparities in data generation mechanisms, differences are likely and necessitate scrutiny to validate the merging of these datasets. We compared the characteristics of RCT data from 5734 diabetic kidney disease patients with corresponding RWD from electronic health records (EHRs) of 23,523 patients. Demographics, diagnoses, medications, laboratory measurements, and vital signs were analyzed using visualization, statistical comparison, and cluster analysis. RCT and RWD sets exhibited significant differences in prevalence, longitudinality, completeness, and sampling density. The cluster analysis revealed distinct patient subgroups within both RCT and RWD sets, as well as clusters containing patients from both sets. We stress the importance of validation to verify the feasibility of combining RCT and RWD, for instance, in building an external control arm. Our results highlight general differences between RCT and RWD sets, which should be considered during the planning stages of an RCT-RWD study. If they are, RWD has the potential to enrich RCT data by providing first-hand baseline data, filling in missing data or by subgrouping or matching individuals, which calls for advanced methods to mitigate the differences between datasets.

Online interviews for qualitative health research in Africa: a scoping review.

Online interviews can be powerful tools in global health research. In this article, we review the literature on the use of and challenges associated with online interviews in health research in Africa and make recommendations for future online qualitative studies. The scoping review methodology was used. We searched on Medline and Embase in March 2022 for qualitative articles that used internet-based interviews as a data collection method. Following full-text reviews, we included nine articles. We found that online interviews were typically conducted via Microsoft Teams, Zoom, Skype, WhatsApp, Facebook Messaging and E-mail chats. Online interviews were used in Africa because of the restrictions imposed by the coronavirus disease 2019 pandemic and the need to sample participants across multiple countries or communities. Recruitment for online interviews occurred online, interviews were characterised by inaudible sounds, the inability to use video options and the challenges of including people with low income and education. We recommend that researchers critically evaluate the feasibility of online interviews within a particular African locality before fully implementing this data collection approach. Researchers may also collaborate with community-based organisations to help recruit a more socioeconomically diverse sample because of the potential of excluding participants with limited internet access.

The impact of electronic versus paper-based data capture on data collection logistics and on missing scores in thyroid cancer patients.

PURPOSE: The purpose of this study was to investigate the impact of the type of data capture on the time and help needed for collecting patient-reported outcomes as well as on the proportion of missing scores. METHODS: In a multinational prospective study, thyroid cancer patients from 17 countries completed a validated questionnaire measuring quality of life. Electronic data capture was compared to the paper-based approach using multivariate logistic regression. RESULTS: A total of 437 patients were included, of whom 13% used electronic data capture. The relation between data capture and time needed was modified by the emotional functioning of the patients. Those with clinical impairments in that respect needed more time to complete the questionnaire when they used electronic data capture compared to paper and pencil (ORadj 24.0; p = 0.006). This was not the case when patients had sub-threshold emotional problems (ORadj 1.9; p = 0.48). The odds of having the researcher reading the questions out (instead of the patient doing this themselves) (ORadj 0.1; p = 0.01) and of needing any help (ORadj 0.1; p = 0.01) were lower when electronic data capture was used. The proportion of missing scores was equivalent in both groups (ORadj 0.4, p = 0.42). CONCLUSIONS: The advantages of electronic data capture, such as real-time assessment and fewer data entry errors, may come at the price of more time required for data collection when the patients have mental health problems. As this is not uncommon in thyroid cancer, researchers need to choose the type of data capture wisely for their particular research question.

Comparative performance of national civil registration and vital statistics systems: a global assessment.

Objective: To assess the current state of the world's civil registration and vital statistics systems based on publicly available data and to propose strategic development pathways, including priority interventions, for countries at different levels of civil registration and vital statistics performance. Methods: We applied a performance assessment framework to publicly available data, using a composite indicator highly correlated with civil registration and vital statistics performance which we then adjusted for data incomparability and missing values. Findings: Globally, civil registration and vital statistics systems score on average 0.70 (0-1 scale), with substantial variations across countries and regions. Scores ranged from less than 0.50 in emerging systems to nearly 1.00 in the most developed systems. Approximately one fifth of the world's population live in the 43 countries with low system performance (< 0.477). Irrespective of system development, health sector indicators consistently scored lower than other determinants of civil registration and vital statistics performance. Conclusion: From our assessment, we provide three main recommendations for how the health sector can contribute to improving civil registration and vital statistics systems: (i) enhanced health sector engagement in birth and death notification; (ii) a more systematic approach to training cause of death diagnostics; and (iii) leadership in the implementation of verbal autopsy methods. Four different civil registration and vital statistics improvement pathways for countries at different levels of system development are proposed, that can constitute a blueprint for regional civil registration and vital statistics strengthening activities that countries can adapt and refine to suit their capabilities, resources, and particular challenges.

A qualitative exploration of forensic pathology service staff perceptions of the implementation barriers and facilitators of manual- and electronic injury mortality surveillance system methods in South Africa.

BACKGROUND: Injury mortality surveillance systems are critical to monitor changes in a population's injury outcomes so that relevant injury prevention responses may be adopted. This is particularly the case in South Africa, where the injury burden is nearly twice the global rate. Regular evaluations of surveillance systems are pivotal to strengthening surveillance capacity, performance, and cost effectiveness. The National Injury Mortality Surveillance System (NIMSS) is an injury mortality surveillance system that is currently focused in Mpumalanga and utilises manual and electronic web-based systems for data collection. This study explored Forensic Pathology Service (FPS) staff perceptions of the implementation barriers and facilitators of manual- and electronic injury mortality surveillance system methods. METHODS: A qualitative study was employed using purposive sampling. Forty-seven participants, aged 29 to 59 years comprising 31 males and 16 females were recruited across 21 FPS facilities that serve the province. The formative evaluation occurred over the November 2019 to November 2022 period. Twelve focus group discussions were thematically analysed to determine emerging themes and patterns related to the use of the system using the WHO surveillance system guidelines as a framework. RESULTS: The key themes concerning the barriers and facilitators were located along WHO attributes of simplicity, acceptability, timeliness, flexibility, data quality and stability. Distinctions between the manual and e-surveillance systems were drawn upon across the attributes highlighting their experience with the system, user preference, and its contextual relevance. With Mpumalanga predominantly rural, internet connectivity was a common issue, with most participants consequently showing a preference for the manual system, even though the electronic system's automated internal validation process was of benefit. The data quality however remained similar for both methods. With program stability and flexibility, the manual system proved more beneficial as the dataset was reported to be easily transferrable across computer devices. CONCLUSION: Obtaining FPS perceptions of their experiences with the system methodologies are pertinent for the enhancement of injury surveillance systems so to improve prospective engagements with the systems. This will facilitate timely and accurate injury mortality information which is vital to inform public policy, and injury control and prevention responses.

Cost and statistical efficiency of posture assessment by inclinometry and observation, exemplified by paper mill work.

Postures at work are paramount in ergonomics. They can be determined using observation and inclinometry in a variety of measurement scenarios that may differ both in costs associated with collecting and processing data, and in efficiency, i.e. the precision of the eventual outcome. The trade-off between cost and efficiency has rarely been addressed in research despite the obvious interest of obtaining precise data at low costs. Median trunk and upper arm inclination were determined for full shifts in 28 paper mill workers using both observation and inclinometry. Costs were estimated using comprehensive cost equations; and efficiency, i.e. the inverted standard deviation of the group mean, was assessed on basis of exposure variance components. Cost and efficiency were estimated in simulations of six sampling scenarios: two for inclinometry (sampling from one or three shifts) and four for observation (one or three observers rating one or three shifts). Each of the six scenarios was evaluated for 1 through 50 workers. Cost-efficiency relationships between the scenarios were intricate. As an example, inclinometry was always more cost-efficient than observation for trunk inclination, except for observation strategies involving only few workers; while for arm inclination, observation by three observers of one shift per worker outperformed inclinometry on three shifts up to a budget of €20000, after which inclinometry prevailed. At a budget of €10000, the best sampling scenario for arm inclination was 2.5 times more efficient than the worst. Arm inclination could be determined with better cost-efficiency than trunk inclination. Our study illustrates that the cost-efficiency of different posture measurement strategies can be assessed and compared using easily accessible diagrams. While the numeric examples in our study are specific to the investigated occupation, exposure variables, and sampling logistics, we believe that inclinometry will, in general, outperform observation. In any specific case, we recommend a thorough analysis, using the comparison procedure proposed in the present study, of feasible strategies for obtaining data, in order to arrive at an informed decision support.

A new acquisition protocol for conducting studies with children: The science camp research experience.

In the last 50 years, the study of brain development has brought major discoveries to education and medicine, changing the lives of millions of children and families. However, collecting behavioral and neurophysiological data from children has specific challenges, such as high rates of data loss and participant dropout. We have developed a science camp method to collect data from children using the benefits of positive peer interactions and interactive and engaging activities, to allow researchers to better collect data repeatedly and reliably from groups of children. A key advantage of this approach is that by increasing participant engagement, attention is also increased, thereby increasing data quality, reducing data loss, and lowering attrition rates. This protocol describes the step-by-step procedure for facilitation of a science camp, including behavioral, electrophysiological, and participatory engagement activities. As this method is robust but also flexible, we anticipate that it can also be applied to different demographics and research needs.

The Hawthorne Effect: Quality and Outcomes in Neurosurgery.

Measure something, and it gets better-this is what is called as the Hawthorne effect (also known as the observer effect). The Hawthorne factory experiments in 1920s were remarkable industrial data collection and analysis exercises that lead to Edwards Deming's quality revolution. The Harvard Medical Practice Study (1991), Leape's "Error in Medicine" (1994), and the Bristol pediatric cardiac report (2001) are among many documents that have revealed the huge gap between best practices and actual medical practice. Alarmed by the poor standards of quality at the most respected institutions, the medical fraternity therefore began visiting facilities in different fields and observing their quality assessment processes. The next leap for neurosurgery is to realize that it is unacceptable to treat patients with no regard for the standard of clinical outcomes. The traditional neurosurgery residency training has long ignored the most important issues of self-assessment, reappraisal, relearning, and measurement of skill and surgical outcomes. However, the experience taken from disparate fields, especially cardiac surgery, may encourage research and progress in measurement and improvement of quality in neurosurgery. Like cardiac surgeons, neurosurgeons must examine and analyze the results of their interventions. The concept of quality measurement is the most important single advance we can make in neurosurgery practice. Meticulous and precise measurement of outcomes will allow future progress of our specialty.

Current state of rare disease registries and databases in Australia: a scoping review.

BACKGROUND: Rare diseases (RDs) affect approximately 8% of all people or > 400 million people globally. The Australian Government's National Strategic Action Plan for Rare Diseases has identified the need for a national, coordinated, and systematic approach to the collection and use of RD data, including registries. Rare disease registries (RDRs) are established for epidemiological, quality improvement and research purposes, and they are critical infrastructure for clinical trials. The aim of this scoping review was to review literature on the current state of RDRs in Australia; to describe how they are funded; what data they collect; and their impact on patient outcomes. METHODS: We conducted a literature search on MEDLINE, EMBASE, CINAHL and PsychINFO databases, in addition to Google Scholar and grey literature. Dissertations, government reports, randomised control trials, conference proceedings, conference posters and meeting abstracts were also included. Articles were excluded if they did not discuss RDs or if they were written in a language other than English. Studies were assessed on demographic and clinical patient characteristics, procedure or treatment type and health-related quality of life captured by RDRs or databases that have been established to date. RESULTS: Seventy-four RDRs were identified; 19 were global registries in which Australians participated, 24 were Australian-only registries, 10 were Australia and New Zealand based, and five were Australian jurisdiction-based registries. Sixteen "umbrella" registries collected data on several different conditions, which included some RDs, and thirteen RDRs stored rare cancer-specific information. Most RDRs and databases captured similar types of information related to patient characteristics, comorbidities and other clinical features, procedure or treatment type and health-related quality of life measures. We found considerable heterogeneity among existing RDRs in Australia, especially with regards to data collection, scope and quality of registries, suggesting a national coordinated approach to RDRs is required. CONCLUSION: This scoping review highlights the current state of Australian RDRs, identifying several important gaps and opportunities for improvement through national coordination and increased investment.

Results and Methodological Implications of the Digital Epidemiology of Prescription Drug References Among Twitter Users: Latent Dirichlet Allocation (LDA) Analyses.

BACKGROUND: Social media is an important information source for a growing subset of the population and can likely be leveraged to provide insight into the evolving drug overdose epidemic. Twitter can provide valuable insight into trends, colloquial information available to potential users, and how networks and interactivity might influence what people are exposed to and how they engage in communication around drug use. OBJECTIVE: This exploratory study was designed to investigate the ways in which unsupervised machine learning analyses using natural language processing could identify coherent themes for tweets containing substance names. METHODS: This study involved harnessing data from Twitter, including large-scale collection of brand name (N=262,607) and street name (N=204,068) prescription drug-related tweets and use of unsupervised machine learning analyses (ie, natural language processing) of collected data with data visualization to identify pertinent tweet themes. Latent Dirichlet allocation (LDA) with coherence score calculations was performed to compare brand (eg, OxyContin) and street (eg, oxys) name tweets. RESULTS: We found people discussed drug use differently depending on whether a brand name or street name was used. Brand name categories often contained political talking points (eg, border, crime, and political handling of ongoing drug mitigation strategies). In contrast, categories containing street names occasionally referenced drug misuse, though multiple social uses for a term (eg, Sonata) muddled topic clarity. CONCLUSIONS: Content in the brand name corpus reflected discussion about the drug itself and less often reflected personal use. However, content in the street name corpus was notably more diverse and resisted simple LDA categorization. We speculate this may reflect effective use of slang terminology to clandestinely discuss drug-related activity. If so, straightforward analyses of digital drug-related communication may be more difficult than previously assumed. This work has the potential to be used for surveillance and detection of harmful drug use information. It also might be used for appropriate education and dissemination of information to persons engaged in drug use content on Twitter.